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Exponent Function for One Helper Source Coding 
Problem at Rates outside the Rate Region 

Yasutada Oohama 


Abstract —We consider the one helper source coding problem 
posed and Investigated by Ahlswede, Korner and Wyner. Two 
correlated sources are separately encoded and are sent to a 
destination where the decoder wishes to decode one of the two 
sources with an arbitrary small error probability of decoding. 
In this system, the error probability of decoding goes to one as 
the source block length n goes to infinity. This implies that we 
have a strong converse theorem for the one helper source coding 
problem. In this paper we provide the much stronger version of 
this strong converse theorem for the one helper source coding 
problem. We prove that the error probability of decoding tends 
to one exponentially and derive an explicit lower bound of this 
exponent function. 

Index Terms —One helper source coding problem, strong con¬ 
verse theorem, exponent of correct probability of decoding 

I. Introduction 

We consider the one helper source coding problem posed 
and investigated by Ahlswede, Korner and Wyner. Two cor¬ 
related sources are separately encoded and are sent to a 
destination where the decoder wishes to decode one of the two 
sources with an arbitrary small error probability of decoding. 
In this system, the error probability of decoding goes to one 
as the source block length n goes to infinity. This implies that 
we have a strong converse theorem for the one helper source 
coding problem. In this paper we provide the much stronger 
version of this strong converse theorem for the one helper 
source coding problem. We prove that the error probability 
of decoding tends to one exponentially and derive an explicit 
lower bound of this exponent function. 

II. Problem Formulation 

Let X and y be finite sets and {(Xt, Tt)}^ be a stationary 
discrete memoryless source. For each t = 1, 2, • • •, the random 
pair {Xt,Yt) takes values in X x y, and has a probability 
distribution 

PXY = {PXYix,y)}^^y^^;^,^y 

We write n independent copies of and 

respectively as 

Yi,Y 2 ,---,Y„ and Y" = Yi, Y 2 , • • •, Y„. 

We consider a communication system depicted in Fig. 1. Data 
sequences Y" and Y" are separately encoded to (/?^"^(Y") 
and ip^'^ (Y") and those are sent to the information processing 
center. At the center the decoder function observes 
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Fig. 1. One helper source coding system. 

{ip^\x^),(p^^\Y^)) to output the estimation Y” of Y". 

(n) fnl 

The encoder functions tpl ’ and are defined by 

^ M^ = {\X - ■ , 

where for each i = 1,2, (= Mi) stands for the range 

of cardinality of <Pi^^. The decoder function is defined 
by 

:MixM2^ V”. (2) 

The error probability of decoding is 

= Pr {y" ^ Y-} , (3) 

where Y" = (Y"), (Y”)). A rate pair (Ri, R 2 ) 

is e-achievable if for any i5 > 0, there exist a positive 
integer no = no{e,d) and a sequence of triples {[pi, 
such that for n > no, 

- log 11 II < R^ + S for i = 1,2, 

n 

For e S (0,1), the rate region 'R-akw{£\pxy) is defined by 
'R'K¥xn{£\pxy) 

= { (i?i, i? 2 ) : {Ri,R 2 ) is e-achievable for pxY } • 
Furthermore, define 

Rakw(pa:y) = Pi RAKw(e|pxY)- 

eg(O.l) 

We can show that the two rate regions Rakw(£| Pxy), £ S 
(0,1) and Rakw(pa:y) satisfy the following property. 
Property 1: 

a) The regions 'JIayxn{£\pxy), £ e (0,1), and Rakw( 
Pxy) are closed convex sets of R^, where 

= {(i?i,i?2) • Ri ^ 0,i?2 ^ 0}. 
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b) 7?.AKw(ebxF) has another form using (n, e)-rate region 
'Ti-AKw{n^£\pxY), the definition of which is as follows. 
We set 

'R'AKAN{n,£\pxY) = {{R 11 R 2 ) '■ 

There exists such that 

^ log 1111 <Ri,i= 1,2, 

Using 7 ^akw(«, £\pxy), RAKwielpxY) can be ex¬ 
pressed as 


when (i?i, R 2 ) ^ R-akw (pxy)- To examine this quantity, we 
define the following quantity. Set 

P(-) ) = 1 - Pl"l ), 

G^^HRi,R2\pxy) 

= , min 

,¥> 2 " ^ 

(l/n) log||¥J<">||<fli,i= 1.2 

G(i?i, i?2bxY) = lim i?2bAY), 

n —^00 

G{pxy) = {{Ri,R 2, G) : G > G{Ri, R2 \pxy)}- 


R-AKwislPXY) = cl u n RaKW {n,£\pxY) ■ 

\ m>l n'>m j 

Proof of this property is given in Appendix 0 It is 
well known that 'R-akyn{pxy) was determined by Ahlswede, 
Korner and Wyner. To describe their result we introduce an 
auxiliary random variable U taking values in a finite set U. 
We assume that the joint distribution of ({7, X, Y) is 

Puxy{u, X, y) = Pu{u)px\u{x\u)pY\x{y\x)- 

The above condition is equivalent to f/ O X o F. Define 
the set of probability distribution p = puxY by 

V{pxy) = {puxY ■.\U\<\X\ + 1,U F}. 


Set 


n{p)= {(i?i,i?2) :i?i,i?2>0, 

Ri>Ip{X-U),R2>Hp{Y\U)}, 

R{pxy) = U n{p). 

P&'P{pxy) 

We can show that the region TZ{pxy) is a closed convex 
set of The rate region TZakw(pxy) was determined by 
Ahlswede and Korner III and Wyner ||2l- Their result is the 
following. 

Theorem 1 (Ahlswede, Korner and Wyner 
Rakw (pxy) = TI{pxy)- 

On the converse coding theorem Ahlswede et al. 0 ob¬ 
tained the following. 

Theorem 2 (Ahlswede et al. SB): ] For each fixed e € 
(0,1), we have 


By time sharing we have that 

Q(n+Tn) f it-Ri + nR2 + mR^ 

\ n + m ’ n + m 

< nG(^HRi,R2\pxY)+mG^"^\R[,R'^\pxY) 

“ n + m 

Choosing R = R' in (IHi, we obtain the following subadditivity 
property on {G^^'> {Ri, R 2 \pxy) } n>l- 


PXY 


G("+™l(i?i,i?2bxY) 

nGl"l (i?i, i?2 |pxY) + (i?i, i?2 |px y ) 

n + m 


from which we have that G^"!(i?i, i? 2 |pxY) exists and satis¬ 
fies the following; 


lim G^^'>{Ri,R 2 \pxy) = inf G^^^Ri, R 2 \pxy). 

1 —>-oo n>l 


The exponent function G(i?i,i? 2 |pxY) is a convex function 
of (i?i,i? 2 )- In fact, from (IHi, we have that for any a € [0,1] 

G(Q;i?i + aR[,aR2 + aiJ^bxY) 

< aG(Ri, R 2 IPXY) + aG(R[,R 2 lpxY)- 


The region G{pxy) is also a closed convex set. Our main aim 
is to find an explicit characterization of G{pxy)- In this paper 
we derive an explicit outer bound of G (pxy) whose section 
by the plane G = 0 coincides with TZakw{pxy)- 


III. Main Result 


'^AKw(e|pxY) = TZ{pxy). 

Gu and Effors 0 examined a speed of convergence for 

(n) 

Pe to tend to 1 as n 00 by carefully checking the proof 
of Ahlswede et al. 0. However they could not obtain a result 
on an explicit form of the exponent function with respect to 
the code length n. 

Our aim is to find an explicit form of the exponent function 
for the error probability of decoding to tend to one as n —00 


In this section we state our main result. We first explain 
that the region TZ{pxy) has two other expressions using 
the supporting hyperplane. We define two sets of probability 
distributions on U xX by 

Psh(pxY) = {p = PUXY : \u\ < \XIU O X o F}, 
Q{Py\x) = {q = quxY : \U\ < \X\,U O X o F, 

Py|x = 9y|x}- 
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For (a,/x) G (0,1] x [0,1], set 

R^^Hpxy) 


= max {fiIp{X;U) + pLHp{Y\U)}, 

p6'Pah(pxy) 

R^^’^Hpxy) 

= min {U Y p)aD{qx\\px) 

^eSiPxix) +a[/24(X; U) + pH,{Y\U)]} , 

(pxy) 

= f| {{RyR2).pRi+pR2>R^^\pxy)}, 

pG[0.1] 


“ ri i — 

pe[o.i] 1 ... I 

— R^°‘'^\pxy) [■, 

a ' 

Rsh{pxY)= Pi 1z^^{pxy) 


aG(0,l] 


A 


P i iRi,R2) ■ pRi + pR2 > 

: /"n 11 ^ -j 

—I6°‘’^\pxy) k 

a J 


aG(0,l], 

pe[o.i] 


For TZ C K^, we set 

TZ — k( 1, 1) = {(a — K,b — k) G : (a, b) G TZ}. 

Then we have the following property. 

Property 2: 

a) The bound \U\ < \X\ is sufficient to describe pxy) 

and pxx)- 

b) For any pxY we have 


TZsh{pxY) = TZ{pxy). 
Furthermore, for any a G (0,ao], we have 

R^^\pxy) - Ci \[^log (c 2 -) 
V a V a/ 

< -^Hpxy) < R^^Hpxy), 

where 


(5) 


a 


( 6 ) 


A 


ao = ao(|A’l,l3^1) = [81og(lA’ll3^1) + l]- 

A 3 


c^ = c,{\xi\y\) = -^\og{\x\\y\)\ 

c2^c.mAy\)^ 


(7) 


iog(|T’||3;|)2' 

The two inequalities of (|6]l implies that for each a G 
(0, aq], 

7^sh(pxv) - Cl.log (c 2 -) (1,1) 

V a V a/ 

^ ’^^^i.PXY) ^ TZshipXY). 

Hence we have 

RshiPXY) = 'R-shiPXY). (8) 


Property |2] part a) is also stated as Lemma |9] in Appendix 
ID Proof of this lemma is given in this appendix. Proof of 
Property |2]part b) is given in Appendix ICl For (a,/r) G [0,1]2, 
define 


IJxIIpX.IJXV'IU 


{x,y\u) 


^ A , <Txix) 

= 1 + /i)a log--- 

px{x) 


-|-Q! 
c{a,^,X) 


'?x|G(a;|M) , . 

p log---h p log 


qxix) 


qY\uiy\u)\ ’ 




= exp{-Ap“£,,,,,,Pa;,ylu)} 


= -logE, 


exp{-Au;J“’£x.,xx|u( 


= - log 


{X,Y\U)] 
X! ^u{u)qx\u{x\u)qY\x{y\x) 




= min 
geQ(p-Kix) 

xy) — aX{pRi + PR 2 ) 

1 + A(1 + /r) 

E(i?i, i?2|PXY) 

= sup F(“’^’^)(pi?i+pi?2bxY). 
(a,p)e[0.1]LA>0 


We can show that the above functions and sets satisfy the 
following property. 

Property 3: 

a) The cardinality bound \U\ < jA”] in Q(py\x) E sufficient 

to describe the quantity (pxv)- 

b) Define a probability distribution q^^ = q\^xY I’A 


q^y{u,x,y) 


q{u, X, y) exp • 

9x|bx.q'xy|u^ ') 

E. 

exp- 

^-Aa;^“P (X,Y\U)'^ 

qx\\px ,qxY\u^ ’ ' \ 

[] 


Then we have 
d 


dA 

dA2 


= {X,Y\U) 

[ (lx\\px,qxY\u^ ’ ' \ 

=-Var„A) „ {X,Y\U) 

’ qx\\px,qxY\u ^ ’ I ^ 


The second equality implies that E a 

concave function of A > 0. 
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c) Define 


P = p{pxy) 

= max Varg 
q&QiPY\x), 


(a.M) 

qx\\px ,qxY\u 


{X,Y\U) 


Since 


0 < 


(“.m) 

qx\\px,qxYiu 


2 


ix,y\u) 


< +00 


of 7 ?.akw(£|pxf) with an asymptotically vanishing deviation 
from TZakw{pxy) = T^{pxy)- The strong converse theorem 
established by Ahlswede et al. El immediately follows from 
this corollary. From Theorem|3]and Property |3] part d) we have 
the following corollary. 

Corollary 2: For each fixed e € (0,1) and for any i5 > 0, 
there exists a positive integer no with 

no = nQ{e,S,\X\, \y\,p{pxY)) 


for {u,x,y) GU X X X y, we have p(pxy) < oo. Then 
for any A G (0,1], we have 


n^‘^’>^'^'>{q\pxY) 


> AEq 




Specifically, we have 


> xR^^’^^\pxy) - \p{j>xy)\^- 

d) For any A > 0, there exists a positive number v = 
z/(A, \X\, |3^|) G (0,1] such that for every r G (0, v\, the 
condition (i?i + r, i ?2 + t) ^ R-Ipxy) implies 


F{Ri, R2 \pxy) > 


p{pxy) ^ 2 ( \ 

2 ^ \‘^p{pxy) ) 


> 0 , 


where g is the inverse function of i?(a) = a + a^, a > 0. 
Property [3 part a) follows from Lemma [TO] in Appendix 
IB] Proof of this lemma is given in this appendix. Proofs of 
Property [3] parts b),c), and d) are given in Appendix iDl Our 
main result is the following. 

Theorem 3: For any > 0, any pxY, and for any 

satisfying 

-logllv^f^ll <Ri,i = 1,2, 

n 

we have 


< 5exp{-nE(i?i,i?2bAY)}- (9) 

It follows from Theorem [3] and Property [3 part d) that 
if (i?i,i? 2 ) is outside the capacity region, then the error 
probability of decoding goes to one exponentially and its 
exponent is not below F[Ri,R 2 \pxy)- E immediately follows 
from Theorem [3] that we have the following corollary. 

Corollary 1: 


G{Ri,R2\pxy) > F{Ri,R2\pxy), 

Q{pxy) C Q{pxy) 

= {{Ri,R2,G)-.G>F{R^,R2\pxy)}- 

Proof of Theorem [3 will be given in the next section. 
The exponent function at rates outside the rate region was 
derived by Oohama and Han fT] for the separate source coding 
problem for correlated sources El- The techniques used by 
them is a method of types El, which is not useful to prove 
Theorem 13 Some novel techniques based on the information 
spectrum method introduced by Han Q are necessary to prove 
this theorem. 

From Theorem [3 and Property [3 part d), we have the 
following corollary, which provides an explicit outer bound 


such that for n > no, we have 

RxKYi{£\PXy) F TZ{pxy) — Kn{l, 1), 

where 



It immediately follows from the above result that for any e G 
(0,1), we have 

R-AKwislPXY) = R-AK-wipXY) = R-ipXY)- 

Proof of this corollary will be given in the next section. 


IV. Proof of the Main Result 


Let be a pair of random variables from the 

information source. We set S = Joint distribution 

Psx-Y^^ of (S',X",y") is given by 

n 

Psx^Y^{s,x^,y'^) =ps\x^{s\x'^) WpxtYAxt,yt)- 

t=i 


It is obvious that S ga AT” ga V". Then we have the 
following. 

Lemma 1: For any rj > 0 and for any 
satisfying 

ilog||(/?|”^|| < Ri,i = 1,2, 
n 

we have 


<PSX" 


qsx^Yr^{S,X^,Y^) 

^ “ log - (G Yn ~ 

^^ 1 , gx"(A") 

0 > - log- - g, 

D ^ 1 , Px"|s(A"|S') 

Ri>- log - - g, 


n px"(A") 


i ?2 > - log 


n pyn| 5 (y"|S') 


+ 46-’^". 


( 10 ) 

( 11 ) 


( 12 ) 


The probability distribution appearing in the right members 
of (fT2l l have a property that we can select them arbitrary. In 
(Uni), we can choose any probability distribution qsx^Y^ on 
SxX^xy^. In we can choose any distribution gx" on 
A"". 

Proof of this lemma is given in Appendix |El From Lemma 
[T] we obtain the following lemma. 
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Lemma 2: For any 77 > 0 and for any ( 79 ^”^ <^ 2 ”^ 
satisfying 

^ log 1111 <Rr,i = 1 , 2 , 

we have 

^ fn \ QXti^t) 

< psx’-yr^ 0 > - > log- - - r], 

[ ^7^1 PxA^t) 

^1 > - V log-^- V, 

n. / V t 


PXti^t) 


1 " 

i ?2 > - V log 

r? 


1 


t-l V*-lt 


+46"”''. 

Proof: In (fTOl i in Lemma [T] we choose qsx"Y" having the 
form 

qsx^YAS,X^,Y^) 

n 

= Ps{S)l[{px,\sx^^^Y4Xt\S,X*-\Y*) 

t^l 

X PYt\SYt-^iYt\S,Y*~^)} . 

In (fTTIi in Lemma [T] we choose gx" having the form 

n 

qx^X^) = X{qxSXt). 




We further note that 


Pxr^\s{X^\S) _ ^ ^,isx^-i{Xt\S,X^-^) 


px^iX'^) 


PXt {Xt 


Py^\s{Y^\S) = \{pY,\SY^~^{Yt\SX-^). 
t^l 

Then the bound (fT^ in Lemma [T] becomes 

Pc"^ > ^ 2 '' , ) < psx^ 




0 > — log 

71 




0 > - V log 

71 * 


PY,\SY^-AYt\SX-^) 
PYt\SXt-'-Yt-^ (Pt l‘5', y‘“l) 

qxtiXt) 


" PXtiXt) 


D ^ 1'^, PXt\sxt-^{Xt\S,X* 1) 

Ri> -y^ log —^- V, 


PXtiXt) 


1 


i?2>-E 

n ^ 


1 


3TT - ^ K 4e 


—nr) 


^ FYt|SF‘-i(Pt|5',P* 

^ In \ 9XtiXt) 

< PSXr^Yr^ < 0 > - > log - - - ri, 

[ ^7^1 PxAXt) 

1 PXt\sxt-^(.Xt\S,X^~^) 


i?l > - V log 

1 " 

i?2 > - V log ■ 

n ^ 


PxAXt) 


1 


+4e-”^ 


— 277 


completing the proof. ■ 

Lemma 3: Suppose that for each t = 1,2, • • •, ?^, the joint 
distribution psx*Y* of the random vector SX^Y* is a marginal 
distribution of psx^Y^- Then we have the following Markov 
chain: 

^ Xt^Yt (13) 

or equivalently that I{Yt] SX*'~^\Xt) = 0. Furthermore, we 
have the following Markov chain: 

Y*-^ O SX*-^ o (Xt, Yt) (14) 

or equivalently that I{XtYt',Y^~^\SX^~A = 0. The above 
two Markov chains are equivalent to the following one long 
Markov chain: 


o sx^-^ o Xt o Yt. 


rt-1 


(15) 


Proof of this lemma is given in Appendix |F] For t = 
1,2,---,?^, let Ut be a random variable taking values in 
Ut = M.\ xX*~^. Define Ut by Ut = (S',From 
Lemmas and |3] we have the following. 

Lemma 4: For any p > 0 and for any 
satisfying 


-log||79^"'^|| < RiA = 1,2, 


we have 


<p,x.r.|o>iy'log5i#|-,, 

qn f J /" / V > 


t=l 


’ PxAXt) 


1 ” 

Ri > - Vlog 

r? 




S 2 > - V log 

n 




pxpuAXtWt) 

PXt (Xt) 

1 

PYt\uAYt\Ut) 


-2p\ +Ae-^A 


The following is well known as the Cramer’s bound in the 
large deviation principle. 

Lemma 5: For any real valued random variable Z and any 
A > 0, we have 

Pr{Z > a} < exp [— (Aa — logE[exp(AZ')])]. 

For for (a, p) G [0,1]^ and A > 0, we define 

r ^ 

(gx-, SIpxy) = - log E 1 


= — log E 


qAA^)^\Xt) ptAxApA^^^^A^tlUt) 


.j.pii+,)t.x^Xt) pTAuA^tlUt) 


ctfiX 


By Lemmas |4] and |5l we have the following proposition. 

Proposition 1: For any (a, p) G [0,1]^, any A > 0, and any 
{pA'^ y satisfying 


log I 11 <RiA = 1,2, 
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we have 

< 5 exp< —n [1 + A(1 + //)] 


-1 


< Psx^Y^ < 0 > (1 + fj.)a 


1 ” 

-Slog 

71 


i=l 


QXt i^t) 
PXti^t) 


1 ” 

ap,Ri > — > log 




PhluS^tlUt) 

Ph(xt) 


apr], 


1 ” 

a/ii ?2 > - log ■ 

n. 


1 


- 2apT] \ + 46“”’' 


-,=i 'pTpv^nut) , 

< PSX^Y" ^o:{p.Ri + PR 2 ) + (1 + p)p 

- q^^+>^^<^{X,)pt{X,)Py%SYt\Ut) 


1 

>- 2 I ^°S 

r7 < ^ 


n 

+46“"’' 


= PSX^Y” 


Uip^ix.) P%^,{X,\U,) 




.PxP”(Xi) 

pS(xM^,^,(y>iv,) 


p%:iu,(x>in) 

> — [a^pRi + PR 2 ) 4“ (1 4“ m)^] ^ 4“ 4e 


(a) 

< exp 


n\ aX{pRi + /ii? 2 ) 4- A(1 + p)p 




+ 4e-”''. 


Step (a) follows from Lemma |5] We choose p so that 
— rj = a\{pRi + /ii? 2 ) 4- A(1 + p)r] 
--n^^'^X(^q^„^S\pxY). 

n 

Solving (fTTl t with respect to r;, we have 

XQi'y.tpX) ^ S\pxy) - a\{pRi + PR 2 ) 
" =-l + A(l + p)-■ 

For this choice of 77 and (fThl l. we have 

< Se"”’’ 

= Sexp-j —n [1 + A(1 + p)]~^ 


^ s\pxy) - aXiflRi + PR 2 ) 

n 


completing the proof. 


Set 


n^^’^’^\pxY) 

= inf min rnax _ —n^°‘''^'^\qxp-, S\pxy)- 


n>l SgMi: Qx^-Qx^- n 


^ _ ^X{jlR^ + PR 2 ) 


Proof: By Lemma 01 for (a,/r) G [0,1]^ and A > 0, we 
have the following chain of inequalities: 


By Proposition [T] we have the folloing corollary. 

Corollary 3: For any (a,p) G [0,1]^, any A > 0, and any 
satisfying 


^ log 1111 < Ri,i = 1,2, 


we have 


< 5 exp < —n 


1 4“ A(1 + p) 


We shall call Q!^°‘'^'^\pxy) the communication potential. 
The above corollary implies that the analysis of 
Pxy) leads to an establishment of a strong converse theorem 
for the one helper source coding problem. In the following 
argument we drive an explicit lower bound of XI^°'’^’^\pxy)- 
For each t = 1, 2, • • •, n, set ut = (s, x^~^) € Ut- For each 
f = 1 , 2 , • • •, n, define 


By definition we have 

exp{-fl(“''^'^)(gxn,^bxF)} 

n 

t—1 

For each t = 1, 2, • • •, n, we define the probability distribution 


(16) 


by 


P^sx^tyl = {p^sxtYUs:x\y*)^ 


P^sx^tyks^x^y^) =Ct Vsx‘F*(s,a;',y*) 


{s,x^,y^)GA4i xX^xy^ 


(17) 


nt- 


qxJ\pXi,PXiYilu, 


{xi,y^\ui) 


where 


Ct= Y PsxtYt{s,x\y^) 


S,X^ 


are constants for normalization. For t = 1, 2, • • •, n, define 

(18) 

where we define Cq = 1. Then we have the following lemma. 
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Lemma 6: For each t = 1,2, and for any (s, 

x*,y*) G M.\ xy*, we have 

xpxtYt\sxt-^Yt~^ {xt,yt\s, a;‘“\ y*“^) 


Lemma 8: 

^ S\pxy) = log $ 
Proof: From (fTSi t we have 


(a,//,A) 




= -logCt +logC't_i. 


( 21 ) 


( 22 ) 


Furthermore, we have 


= P^sx^t-iYt-i{s,x*-^,y*-^) 

s,x^ ,y* 

xpxtYt|sx‘-iY‘-i {xt,yt\s, 2/*“^) 

(cct, t/tlrtt). 
WtUPXt^PXtYtlUt^ ’' 

Proof of this lemma is given in Appendix |G] Define 

Puf'^\ut) = P^sxt-^ ('S’ 


(19) 


From Lemmas |3] and |6] we have the following lemma. 
Lemma 7: 

= XI Puf’^\^t)pxpUt{Xt\ut)pYpxAyt\xt) 

ut,xt,yt 

xAa,fi\) (xt,yt\ut). 

•'gxt\\pxt,PxtYt\Ut^ Eiyii t; 

Proof: By Lemma 12 we have 


( 20 ) 


PXtYt|SX*-iY‘-i (2;*, 2/t|s, X* ^,2/‘ 

= PXt\Sxt-^Yt-^[xt\s,x*'~^ ,y*~^) 
xpYt|sx*Y‘-i(2:t|s,a;‘ ^,y‘ 

= Pxt|sx*-1Y*-1 (a^t |s, x*~'^)pYpxt {yt\xt) 

for (s, x*^,y*) G S X X* X and for f = 1, 2, • • •, n. 

Then by Lemma |6] we have 

= X P^sx^t%t-i{s,x*-\y*-'^) 
s,x* ,y* 

xpxtY|sx*-iY*-i {xt,yt\s, 2/‘“^) 

= X pfx^'%t-ds,x*~\y*~^) 

s,x* ,y^ 

^PXt\SX*-^{xt\s,X^~^)pYpXt{yt\Xt) 

= X PsT^-hs,x*~^) 

s,x*,Vt 

XPXt\SX*-^ixt\s,X*~^)pYpXtiyt\xt) 

completing the proof. ■ 

We have the following. 


Furthermore, by definition we have 

n^‘^’>^’^\qx^,S\pxY) = -logC„,C'o = 1. (23) 

From (I 22 I 1 and ( l22 l, (1211 1 is obvious. ■ 

The following proposition is a mathematical core to prove 
our main result. 

Proposition 2: For any a > 1, /r, A > 0, we have 
^(a,P,A)(p^Y)>r!(“’^’^^(pXY). 

Proof: Set 

Qn(PY|x) = {q = quxY : \U\ < \Mi\\X'--^\\y^-\ 

qY\x = py\x,u o X o y}, 

(^Pxy) = min n''°‘’^’^\q\pxY)- 

q&QniPY\x) 

Let Ut be random variables taking values in Adi x X*~^. We 
choose qu^ so that 

qutiut) = (24) 

and choose qxtYt\Ut ’^^at 

qxtYputi^t^ytlut) = px,Yt\utixt,yt\ut). (25) 

Furthermore we have 

(a) 

qYtixti/t(ytlxt,ut) = Prpxjytlxt) = qYpxAytlxt)-(26) 
Step (a) follows from Lemma |3] The equation (l26T l imply that 

qt = qUtXtYt G Qn(PY\x)- 

Hence we have the following chain of inequalities; 

^(«.P.A) 

- X <lUtiut)pxpUtixt\ut)pYt\Xtiyt\xt) 

ut,xt,yt 

xf^°‘fi^^ (xt,yt\ut) 

■'qXt\\pXt,PXtYt\Ut^ 

= X qutiut)pxputixt\ut)pYt\Xtiyt\xt) 

ut,xt,yt 

xf^°‘fi^'^ {xt,yt\ut) 

■'qxf\pxt,qxtYt\ut' ' 

= exp 

> exp |H(f’'"’^)(pxY)| = exp |h(“’'"’^)(pxy)|-(27) 

Step (a) follows from Lemma |7] Step (b) follows from (l25T l. 
Step (c) follows from qt G Qn(PY|x) the definition of 
Ci^’^’^\pxy)- Step (d) follows from Lemma fTOl in Appendix 
m To prove this lemma we bound the cardinality \U\ appearing 
in the definition of (i\^’^’^\pxy) to show that the bound 
\U\ < \X\ is sufficient to describe ^(^’^’^^(pxy)- Proof 


of Lemma [TO] is given in Appendix |B] Hence we have the 
following: 


inax _ , S\pxy) 


qx-^-qx^— n 
n?=i qxt 


> -n^^’>^’^Hqx^,s\pxY) = - Viogch 

Tt n < ^ 








(28) 


Step (a) follows from Lemma |8] Step (b) follows from dZTl i. 
Since (l28T l holds fo any n > 1 and any psx^Y" satisfying 
S O X" o F", we have that for any a > 1 and for any 
positive p, A, 

n^^’^’^HpxY)>n^’^’^’^\pxY). 

Thus, Proposition 12] is proved. ■ 

Proof of Theorem^ We have the following: 

- log ^ ^ ^ 


(^) n^^’^^’^\pxY) - aXjflRi + pR2) 

1 + A(1 + /x) 

(b) - aXjfLRi + pR2) 

1 + A(1 + /x) 

Step (a) follows from Corollary [3] Step (b) follows from 
Proposition |2| Since the above bound holds for any positive 
a, /i, and A, we have 


1 




> F{Ri, R2 \pxy)- 


for any n > mo(e). Fix any b > 0. We take a positive number 
i' = i'{S, l-Tl, |F|) S (0,1] appearing in Property [3] part d) and 
set 


t„ = < 2p{pxy)'& 


np(pxY) 


1 » 8 (*) 




3+T 


(32) 


Since g is an inverse function of d, (13^ is equivalent to 

9 (iJSo) = (33) 

We take a sufficiently large positive integer mi so that we 
have Kn < r' for n > mi. Set uq = max{mo, mi}. We claim 
that for n > Uq, we have (i?i + k„, i ?2 + ^n) S R-ipxY)- To 
prove this claim we suppose that (i?i + Kn*, R 2 + Kn *) does 
not belong to TZ{pxy) for some n* > no- Since TZ{pxy) is a 
closed set, there exists a positive number r > Kn* sufficiently 
close to Kn* such that 

Kn* <T < V, 

(i?i + r, i?2 + r) ^ TZ{pxy)- 

Then we have the following chain of inequalities: 


5 exp [-n*F(iT!i, i^abxry)] 

n*p{pxY) ^2 
2 


(a) 

< 5 exp 


Thus (|9]l in Theorem [3] is proved. ■ 

Proof of Corollary |2|- To prove this corollary we use the 
following expression of R-AKwislPxY) stated in Property ID 
part b): 

F-akw {^Ipxy) = cl u n F-AKwin, e\pxY)\ ■ (29) 

\ m>l n>m j 

We assume that 

(i?i,i?2) € u n T^AKXf{n,e\pxY)- (30) 

m>l n^m 

Then there exists a positive integer mo = rno{e) and some 
\ <p'^\ '(/)("■)) }n>mo such that for n > mo(e), we have 

ilog||v?^”^|| < = 1,2 

n 

Then by Theorem [3| we have 


< 5 exp 

(c) r 

= 5 exp 
= 1 — e. 


9 l,2p(pxv)) 


n*p{pxY) ^ 

2 9 y2p{pxY) 

n*p{pxY) 2 


Cp{px\ 


log 


1 — e 


(34) 


< 5exp{-nF(i?i,i?2bxY)} 


(31) 


Step (a) follows from Property[3]part d). Step (b) follows from 
Kn* < T. Step (c) follows from from ( l33l l. The bound ( |34| ) 
contradicts dJTT ). Hence we have (i?i + K„, i? 2 +«^n) ^T^{pxy) 
or equivalent to 

(i?l, i?2) S TZ{pxy) — Kn(l, 1) 

for n > no- Recalling the first assumption ( l30l ) on {Ri^Rf), 
we obtain 

u n TlAYYN{n, e\pxY) Q TI{pxy) — Kn(l, !)■ (35) 

m> 1 n'>m 

Taking the closure of both sides of dTSl l. using (|29j), and 
considering that 7 ?.(pxy)—k„( 1, 1) is a closed set, we have 
that for n > no, 

T^AKwi^lPXx) 

=4u n T^AKYi{n, e\pxY) \ ^'^(pxy) — 1), 

\ m>l n'>m J 

completing the proof. ■ 























9 





Fig. 2. One helper source coding system investigated by Wyner. 


b) We set 

'RM^{n,e\pxYz) = {(^i, ^ 2 ,-R 3 ) : 

There exists such that 

ilog||(^|”^|| < Ri,i = 1,2,3 

which is called the (n,£:)-rate region. Using TZwin, 
£\pxyz), R-wislPxYz) can be expressed as 


V. One Helper Problem Studied by Wyner 

We consider a communication system depicted in Fig. 2. 
Data sequences X"^, F”, and Z”, respectively are separately 
encoded to and The encoded 

data and and are sent to the informa¬ 

tion processing center 1. The encoded data p^T^\x^) and 
^j-g ggjjj. J.Q jjjg information processing center 2. 
At the center 1 the decoder function observes 
^0)(yn)) jQ output the estimation F” of F". At the center 
2 the decoder function observes (X"), 
to output the estimation F" of F". The error probability of 
decoding is 

= Pr{F"^F"orF"^F"}, 
where F” = (^"), (^”)) and F” = 

p[^\x-\pi^\z-)). 

A rate triple (i?i, i? 2 , R 3 ) is e-achievable if for any b > 0, 
there exist a positve interger uq = no{e,S) and a sequence 
of three encoders and two decoders functions 

4>^'^'^)}n>no such that for n > no(e, b), 

+ (5 for i = 1,2,3, 
n 

The rate region 'Rxj{s\pxyz) is defined by 

RYl{e\PXYz) = {(Ri,R2,R3) '■ 

(i?i, i? 2 , 1 ^ 3 ) is e-achievable for pxYz}- 


'R-w{£\pxyz) = cl u n ebxYz) ■ 

\ m>l n^m j 

It is well known that RxxipxYz) was determined by Wyner. 
To describe their result we introduce an auxiliary random 
variable U taking values in a finite set U. We assume that 
the joint distribution of (t/, X, F) is 

Puxy{u, X, y, z) = Pu{u)Px\u{x\u)pYz\x{y, z\x). 

The above condition is equivalent to U O X o FF. Define 
the set of probability distribution onU xX xy xZby 

ViPXYz) = {p = PUXYZ ■ \ll\ < \X\ + 2, 
U^XoFF}. 

Set 

T^ip) = { (^ 1 1 , -R 3 ) : Ri,R2,R3 > 0, 

Ri>Ip{X-U),R2>Hp{Y\U), 

R3 > HpiZ\U)}, 

nipxYz) = U nip). 

PS:'P{pxY z) 

We can show that the region n{pxYz) is a closed convex set 

ofM3 . 

The rate region RwiPxYz) was determined by Wyner 
El. His result is the following. 

Theorem 4 (Wyner m: 

RwipXYz) = nipxYz). 


Furthermore, define 

n^ipXYz) = P) l^w(e|PXFz). 

eG(O.l) 

We can show that the two rate regions 7^w(£^| Pxyz), £ S 
(0,1) and n^ipxYz) satisfy the following property. 
Property 4: 

a) The regions 7^w(ebxYz), e £ (0,1), and 7^w( Pxyz) 
are closed convex sets of K.^. 


On the strong converse theorem Csiszar and Korner ii 
obtained the following. 

Theorem 5 (Csiszar and Korner Si^): For each fixed e € 
(0,1), we have 

RwislpxYz) = n{pxYz). 

To examine a rate of convergence for the error probability 
of decoding to tend to one as n —00 for {Ri, R2, R3) ^ 
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TZw{pxyz), we define the following quantity. Set 

G(")(i?i,i?2,i?3bx Yz) 

= .min 

(¥>r .vr'.¥=r A 

^(n)^ 0 (")): 

(1/n) log||(p'”''|| 

<iti,i=l,2,3 

G(i?i,i?2,.Rabxyz) = lim GA)(i?2,i?2,.R3bxyz), 

n—^oo 

G{pxyz) 

= {(i?i,i?2,i?3,G) : G > G(i?i,i?2,i?3bxYz)}. 


By time sharing we have that 

nR2YfTiRo nR2Y'^R'o 


^(n+77i) f nR\-\-mR'^ 

^ n+m ’ ■ 




< 


n+m ’ n+m 

nGA) (i?i, i?2, i?3bxYz) + mG(™) , i??, \pxYz) 


n + m 


(36) 


Choosing i? = i?' in (l36l l. we obtain the following subaddi¬ 
tivity property on {G(A(i?;^^ i? 2 , i? 3 bxYz) }n>i: 

GA+-)(i?i,i?2,i?3bx Yz) 

^ nG^'^\Ri,R2, RsIpxyz) + iRuR2, RsIpxyz) 

~ n + m ’ 

from which we have that G(i?i, i? 2 , ^ 3 |bPi, MA) exists and 
satisfies the following: 


G(i?i, i?2, RsIpxyz) 

= inf G^'^\Ri,R2,R3\pxyz)- 

n>l 

The exponent function G{Ri,R 2 ^ R^Ipxyz) is a convex func¬ 
tion of (i?i,i? 2 ,i^a). In fact, by time sharing we have that 

^(n+m) f nRi-\-mR\ nR 2 +mR'^ nR 2 +mR'^ \ 

y n+m ’ n+m ’ n+m PXYZ J 

^ nG(-^HRi,R2,R3\px yz) + mG^"^'> {R[, R 2 , R'sIpxyz) 

~ n + m ’ 

from which we have that for any a G [ 0 , 1 ] 


For (a, n, 7 ) G (0,1) x [0,1]^, set 
R^^’^\pxyz) 


A 


= max {p,Ip{X;U) 

peVahiPXYz) , , , . 

+p(7i^p++)+7i?p+|G))}, 

R^^’^’^Hpxyz) 

= min {(1-I-/r)a£)((jx|bx) + Q:[/2/,(X; G) 

.ea(Pv.ix) ^ jH,{Z\U))]} , 

?^sh(PXYz) 

= {{Ri, R 2 , R 3 )'■ pRi + p{iR 2 + jRa) 

(p.7)e[o,i]2 

^R^^’-^Hpxyz)}, 

^ih^bxYz) 


— i (.Ri+2 , Fia) : pi?i-b/i(7i?2 + 7 ^ 3 ) 

(p.7)e[o.i]=) 

> —R^ ’^’^\pXYz) k 

a J 

■^shbxY) = Pi Ri^\pXYz) 


aG(0.1] 


— P i (.Ri+2+3) : fi-Ri +/z( 7 .R 2 + 7^3) 

Y^,izfn Ilk ^ 

a J 


ae(0.1]. 

(p. 7 )e[o.i] 


Then we have the following property. 

Property 5: 

a) The bound \U\ < \X\ is sufficient to describe R^>^\ 
pxYz) and .RA)( pxYz)- 

b) For any pxYZ we have 


■^shbxYz) = R{PXYz)- 
Furthermore, for any a G (0, ao]j we have 


(37) 


(pXYz) - Cl log (c2 


<—P^’^\pxYz) < Rl^’(pxYz), (38) 


?(m) / 


G{aRi + ai?'i, ai?2 + ai?b cti?3 + aR'^\pxYz) 

< aG(i?i,i?2,7?3bxYz) + aG{R'i,R'2,R'^\pxYz)- 

The region G{pxyz) is also a closed convex set. Our main 
aim is to find an explicit characterization of G{pxyz)- In 
this paper we derive an explicit outer bound of Q {pxYz) 
whose section by the plane G = 0 coincides with TZw{pxyz)- 
We first explain that the region TZ{pxYz) has two other 
expressions using the supporting hyperplane. We define two 
sets of probability distributions onG xX xj^ xZby 

RshiPXYz) = {p = PUXYZ ■■ |G| < \X\, 

U -H- X -h-YZ}, 

2(pyz|x) = {9 = quxYZ ■ \u\ < IfFl, 

Fyz|x = 9yz|X: U X YZ}. 


where ao = ao(|A|, +|, \Z\), ci = ci(|A’|, +|, \Z\), and 
C 2 = C 2 (|A’|,+|, |Z|) are some suitable constants. The 
two inequalities of (l38]) implies that for each a G {0, ao], 

R-shiPXYz) -Cl\ - log (C2- J (1, 1, 1) 

V a V a/ 

^ '^sh\pXYz) ^ 7^sh(PXYz)- 
Hence we have 

■^shbxYz) = R-shipXYz)- (39) 

Property 6: 

RshipXYz) = Rsh{pXYz) = Tl{pXYz)- 
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Since the proof of Property [ 6 ] is quite similar to that of 
Property [21 we omit it. For (a,//, 7 ) G [0,1]^, define 


Lu M {x,y,z\u) 

qx\\px,qxYz\u^ ^ 

A qx{x) 

= (1 + fi)a log-+ a 


+fi 7 log- 


' Px{x) 

1 


log 


qx\u{x\u) 

qx{x) 


7 log ■ 


qY\u{y\u) ' ' gz|(7(^|u)yj ’ 


= — log Eq exp I—Aw 


= - log 


^(“.P,7) 

<ix\\px,qxYz\u 

(a,/i,7,A) 


(x,r,z|i7)}' 




'.,x,y,z 


= min 
gGQ(pi'zix), 

F<^‘^’^^’'^’^\a[Ri+fi{jR2+jR3)]\pxYz) 

A »(“’^-7-^)(pxfz) - a[i^i + piiR2 + 7 ^ 3 )] 

1 + A(1 + /x) 

F(i?i, i?2, -R3 bxrz) 

= sup +7i?3)]|pxYz), 

A>0, 

(a,P,7) 

G[0.1]^ 

Q{pxyz) 

= {(i?i, i?2, i ?3 , G) : G > F(i?i, i?2, i ?3 bxYZ)} . 

We can show that the above functions and sets satisfy the 
following property. 

Property 7: 

a) The cardinality bound \U\ < \X\ in Q{pyz\x) is suffi¬ 
cient to describe the quantity ^'^°''^'"^'^\pxYz) 

b) Define a probability distribution = q^^xyz I’A 

q^^\u,x,y,z) 


A 


q{u,x,y,z) exp. 

-^4x\\px,qXYZiU^^^y’^ 

u)} 

Eg 

exp. 

—Aw 

'Ztt„r„PYY.Z\U)] 

[] 



c) Define 

P = P{PXYZ) 


A 

= max 
geC(pvix). 


Varo 


iX,Y,Z\U) 

qx\\px,qxYziu'^ ’ ’ ' ^ 


Since 


0 < 


w |7 ix.y, z\u) 

qx\\px,qxYz\u^ ° ' ' 


< +00 


for {u,x,y,z) GUxX xyxZ, we have p{pxYz) < oo. 
Then for any A G (0,1], we have 

f^(“’'^’7’^)(<?bxYx) 

Specifically, we have 

J^(a.M.7.A)(p^Yz) > \R^^'^\pxYz) - \P{PXYZ)\^. 

d) For any i5 > 0, there exists a positive number v = 
ITFI, | 7 |, |.Z|) G ( 0 , 1 ] such that for every positive 
T G ( 0 , 1 /], the condition (i?i + t, i ?2 + x.R^ + r) ^ 
F-ipxYz) implies 

F{Ri,R2,R3\PXYz) > 

Since the proof of Property |7] is quite similar to the proof of 
Property [3 we omit it. 

Our main result is the following. 

Theorem 6: For any i?i,i? 2 , R 3 > 0, any pxYZ, and for 
any (v?i”\ P 2 ^\ P 3 ^\ satisfying 

i log I 11 <Rt,i = 1,2,3, 

we have 

< 7exp{-uE(i?i,i?2,l?3bxYz)}- 

It follows from Theorem | 6 | and Property |7| part d) that if 
(i?i, i? 2 , f? 3 ) is outside the capacity region, then the error 
probability of decoding goes to one exponentially and its 
exponent is not below F{Ri, R 2 , RsIpxyz)- It immediately 
follows from Theorem [3 that we have the following corollary. 

Corollary 4: 


Then we have 


^(<^,PAP)^q\pXYz) 

dA 


= EgCA) 


(x,y,z|t/) 

gxllpx.gxyzw^ ' \ 




= -Varg(x) 


9x||px,gxyz|[/ 


{X,Y,Z\U) 


The second equality implies that {q\pxYz) is 

a concave function of A > 0. 


G(i?i, i?2, I? 3 bxYz) > F{Ri,R 2 tR 3 \pxYz), 

G{pxyz) F G{pxyz) 

= {(i?i, i?2, i? 3 , G) : G > F(i?i , i?2 , -Ra Ipxyx)} ■ 

Proof of Theorem |3 will be given in the next section. 
The exponent function at rates outside the rate region was 
derived by Oohama and Han Q for the separate source coding 
problem for correlated sources 0 . The techniques used by 
them is a method of types 0 , which is not useful to prove 
Theorem [3 Some novel techniques based on the information 
spectrum method introduced by Han ||9| are necessary to prove 
this theorem. 
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From Theorem |6] and Property Q part d), we have the 
following corollary, which provides an explicit outer bound 
of 7^w(ebxFz) with an asymptotically vanishing deviation 
from Ti-wiPxYz) = T^{pxyz)- The strong converse theorem 
immediately follows from this corollary. 

From Theorem |6] and Property [T] part d) we have the 
following corollary. 

Corollary 5: For each fixed e G (0,1) and for any 6 > 0, 
there exists a positive integer no with 

no = no{e,6, \X\, |3^|, \Z\,p{pxYz)) 
such that for n > no, we have 

Ti-wi£\PXYz) ^ Ti-iPXYz) — 1 , 1 ), 

where 



It immediately follows from the above result that for any e € 
(0,1), we have 

'R-wislPxYz) = 'R-wipxYz) = TI{pxyz)- 


Here we assume that there exists a pair (i?i, i? 2 ) belonging to 
T^akw{£\pxy) such that 


(i?l, i?2) ^ cl 


[J 'KAKwi''^^^\PXY) 


(42) 


I m>l 


Since the set in the right hand side of (l42l) is a closed set, we 
have 


(i?i + 6, R 2 + 5) ^ cl i 2;akw(’^i j (43) 


L m>l 


for some small (5 > 0. On the other hand we have {R+5, R 2 + 
(5) £ TZ^^l^yf{e\pxY), which contradicts (HTI) . Thus we have 

IJ RxKwim, e\pxY) 

m>l 


c 7^AKw(ebxF) ^ cl 


IJ RxKwi'rn, £\pxy) 


%>i 


(44) 


Note here that 7?.AKw(e|pxF) is a closed set. Then from (l44l) . 
we conclude that 


Appendix 

A. Properties of the Rate Regions 

In this appendix we prove Property [T] Property [T] part a) 
can easily be proved by the definitions of the rate distortion 
regions. We omit the proofs of this part. In the following 
argument we prove part b). 

Proof of PropertyUlpart b: We set 

‘KAKwi^y ^\PXy) = P) R-AKwin, £\pxy)- 

n'>m 

By the definitions of e\pxY) and 'R-AKXf{£\pxY), 

we have that ^ 7^AKw(ebAy) form > 1. 

Hence we have that 

[J E:AKw(™i^bxy) ^ '^AKw(ebxy)- (40) 

m>l 

We next assume that (i?i,i? 2 ) £ '^AKw(e|PAy)- Set 

^AKw(^|P^y) 

= {(i?i + (5, i?2 + <5) : (i?i, i?2) £ '^AKw(e|pxy)} 

Then, by the definitions of 7^AKw(i^:e \pxy) and 7 ?.akw( 
£\pxy), we have that for any 5 > 0, there exists no(e, (5) such 
that for any n > no(e, S), 

{R + S, R 2 + S) G 7^akw(i^, £\pxy), 


which implies that 


'^AKw(^|P^’t") ^ U RAKwin, e\pxY) 

n>no (£,<5) 

= ^ akw (^ o (< 5 )i £\pxy) 


C cl j pi ^AKw(’^>£bxy) • 


1>1 


'^akw(£^|W^) = cl MJ ^AKw(”T-i£bxy) 


i>i 


^ U n ^AKw(l2,£bAy) 

completing the proof. 


i m>l n>m 


B. Cardinality Bound on Auxiliary Random Variables 
We first prove the following lemma. 

Lemma 9: 

R^^\pxy) = min {fiIp{X- U) + pHp{Y\U)} 

P&P(pxy) 

= R‘''^Hpxy)= min {flIpiX;U) + pHp{Y\U)}. 

P^VshiPXY) 

Proof: We bound the cardinality \U\ of U to show that the 
bound \U\ < \X\ is sufficient to describe R^^\pxy)- Observe 
that 

px{x) = '^pu{u)px\u{x\u), (45) 

flIp{X-,U) + pHp{Y\U) = ^ puiu)Tr(j)x\ui-\u)), (46) 

u^U 

where 

-^(.Rxiui-lu)) = Px\u{x\u)pY\x{y\x) 

{x,v)(iXy.y 

X! PY\x{y\x)px\u{x\u) 

.xCX 

For each u G U, 'tt{px\u{-\u)) is a continuous function of 
Px\u{'W)- Then by the support lemma, 

\U\<\X\-\ + l = \X\ 


X log ■ 


Px\ui^\^) 

Pxi^) 



(41) 
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is sufficient to express jA"! — 1 values of ( 1451 ) and one value 
of®. ■ 

Next we prove the following lemma. 

Lemma 10: For each integer n > 2, we have 

= mm 

q—quxv-U-i-^X-ir^Y, 
qY\X=PY\X , 

\u\<\Mi\\xr-^ 

= min n^‘^’>^’^\q\pxY) 

q—quxY-U -f-j-x-o-y, 
qY\X=PY\X: 

\u\<\x\ 

= n^^’>^’^\pxY). 

Proof: We bound the cardinality \U\ of U to show that 
the bound \U\ < \X\ is sufficient to describe Clif’^'^\pxY)- 
Observe that 

Qxix) = ^ quiu)qx\uix\u), (47) 

u^lA 

exp 

= <lu{u)Il{qx\ui-\u)), (48) 

u^lA 

where 


We prove 'R-s\i{pxy) ^ T^{pxy)- We assume that {Ri,R 2 ) ^ 
TZ{pxy)- Then by LemmafTTl there exist e > 0 and p* G [0,1] 
such that for any (i?i,i? 2 ) € TZ{pxy)^ we have 

lI*Ri + p*R 2 < p*Ri + p*R 2 - e. 

Hence we have 

p*Ri + p*R 2 < min {p*Ri + p*R 2 } — e 

(RuR2)(^n(pxY) 

= min {p*Ip{U- X\Y) + p*Hp{Y\U)} - e 

pERipxY) 

< min {p*Ip{U;X)+p*Hp{Y\U)}-e 

peVshipxY) 

= R^^'Hpxy) - e. (49) 

Step (a) follows from the definition of TZ{pxy)- The inequality 
® implies that {Ri,R 2 ) f. IZshipxY)- Thus IZshiPxY) Q 
R-iPxx) is concluded. We next prove TZ{ pxy) Q R-shiPxY)- 
We assume that (i?i, R 2 ) G 7Z(pxy)- Then there exists q G 
R (pxy) such that 

Ri>Iq{X-,U),R2>H,{Y\U). (50) 

Then, for each p > 0 and for G TZ{pxy), we have 

the following chain of inequalities: 


n(gx|(7(») = <ix\M^\u)qY\x{y\x) 

ix,y)GXxy 

For each u G U, n(gx|c/(‘|u)) is a continuous function of 
qx\u{‘W)- Then by the support lemma, 

\U\<\X\-l + l=\X\ 

is sufficient to express jA"! — 1 values of (l47T i and one value 
of®. ■ 

C. Supporting Hyperplain Expressions of TZ{pxy) 

In this appendix we prove Property |2] part b). We first 
prepare a lemma useful to prove this property. 

From the convex property of the region TZ(j)xy), we have 
the following lemma. 

Lemma 11: Suppose that {Ri,R 2 ) does not belong to 
TZ{pxy)- Then there exist e,/r* > 0 such that for any 
(i?i,i? 2 ) € 1Z{pxy) we have 

p{Ri — Ri) + p*{R 2 ~ R 2 ) — e > 0. 

Proof of this lemma is omitted here. LemmafTTIis equivalent 
to the fact that if the region 1Z{pxy) is a convex set, then for 
any point {Ri,R 2 ) outside the region 1Z{pxy), there exists 
a line which separates the point {Ri,R 2 ) from the region 
1Z{Pxy)- Lemma [TT] will be used to prove Property |2] part 
b). 

Proof of Property]2\part b): We first recall the following 
definitions of V*{pxy) and V{pxy)'- 

V{pxy) = {puxY :\U\<\X\ + l,U X Y}, 
Vst^ipxY) = {puxY :\U\<\X\,U GgX GG r}. 


pRi + pR2 > pIg{X; U) + pHq{Y\U) 
> min [pIq{X-,U) + pHq{Y\U)\ 

qeVipxY) 

= R^>^Hpxy). 


Step (a) follows from (ISOl l. Hence we have 1Z{pxy) ^ 
RshiPxx)- We next prove the second inequality of (|6]l in 
Property |2] We have the following chain of inequalities. 




XY) 


> min [pIg{X;U) + pHg{Y\U)] 

qe'PahipxY) 


(a) 

= mm 
qGVIpxy) 


> min 
qgQIpy \x) 


(1 + p)a 


DiqxWpx) 


+fiI,(X-U) + liH,(YlU)} 

a 

+ pIg{X-U)+pHgiY\U)} 


= -R^°‘'^\pxy)- 


Step (a) follows from that when q G Rsh(pxY), we have 
D{qx\\px) = 0. We finally prove the first inequality of (|6]l in 
Property |2] Let 


qa,p — qUXY,a,p ^ Q{PY\x) 

be the probability distribution attaing the minimum in the 
definition of R!^°‘'^^\pxY)■ Let qa,^^ = quxYZ,a,p be a 
probability disitribution with the form 

qUXY,a,p.{u,X,y) = qf^x,a,pif^\^)Px{x)pY\x{y\x). 
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By definition, we have qa,ti & 'Pshipxv)- Then we have the 
following chain of inequalities. 

= aD{qx^^J\px) 

< {1 + p)aD{q*^^^J\px) 

+a[flIg.^jX;U) + pH,.JY\U)] 

= R^^’^\pxy) 

< (1 + p)a[D{qx,a,fj.\\px) 

+a[flIg^jX;U) + pH^^jY\U)] 

= a[fLlg^jX;U) + pH^^jY\U)] 
^alogdA-llJ^I). (51) 

For simplicity of notation we set ^ = logd-Tj |3^|). From (ISTT i 
we have 

<e“- (52) 

By the Pinsker’s inequality we have 

< D{q*^J\q^,^) (53) 

From (l52]) and (l53]) . we obtain 

IK^-fc.Mlli<y2e|<^- (54) 

for any a G (0,q;o] with 

ao = (8^ + l)-i = [Slogd-TllJ^D + lJ-d (55) 

The bound (l5^ implies that for any A C {U^ X,Y, Z}, we 
have 

I ^ 1 

~ qA,a,fj.\\l ^ y — 2 

Then for any a G (0, q;o]> we have the following two chains 
of inequalities; 


\I,,^JU-X)-I^^JU;X)\ 

< iHg.^ jU) - Hg^jU)\ + \H,.^ JX) - H^^JX)\ 
+ \H,,^jUX)-Hg^jUX)\ 


(a) 

< 



log 



+ log 


mx\)] 

|iT,j^^(r|;7)-iJ^_(F|c/)| 
< \H,. JYU) - H^^JYU)\ 


(b) 

< 



log 




■ lA-l 


(57) 


+ log 



< 


2 ^“log 

a 



(58) 


Steps (a) and (b) follow from ( l56b and LEMMA 2.7 in Section 
1.2 in Csiszar and Korner |[8|. Combining ( |57] | and ( fSSl l, we 
have 


flI,.^jX;U) + pH,,^jY\U) 

-{flk^jX;U) + pH^^jY\U)) 
< fl\I,,^jX;U)-Ig^jX;U)\ 
+P\H,,^JY\U) - H^^JY\U)\ 

/■ B — ti. 

( 


< \/2^“log 

a 


a 


\u\w\y\^ 


< \l2^-log- 




(A „ 

< W2^-log 


a . 


( 


a 

\2^ 


|A’d|3;| 


= Cl. 


■log (c 2 -) • 
V a/ 


(59) 


Step (a) follows from the cardinality bound \U\ < \X\ we have 
for 7a,n £ Q{Py\x)- Then we have 

a 


-R^^’^Kpxy) 


= {1 + P)-D{q*x\\px) 


a 


+I,.^jX-U)+pH,.^jY\U) 

hi 


> I,,_lx-U) + pH,,^jY\U) 


> Iq^A^-,U) Y pHq^^^{Y\U) - Cl log (c 2 ^) 

(b) 

> min {Iq{X]U) + pHq{Y\U)} 

qeVshipxv) 

-Cl. log (c 2 -) 

V a \ a/ 

= R^^\pxy) - Cllog (c2-) . (60) 

V a \ a/ 

Step (a) follows from ( |59] ). Step (b) follows from that q^^^ 
e Q.{py\x)- ■ 


D. Proof of Property\^parts b), c), and d) 

In this appendix we prove Property [3 b),c), and d). 

Proof of Property\^),c), and d): We first prove part b). For 
simplicity of notations, set 

a = iu,x,y),A = {U,X,Y),A = U X X xy, 
=9{a), 

Then we have 


We set 


e(A) 


log 


aeA 


q^y{u,x,y,z) 


= P^I\a). 
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Then (a), a S A has the following form: 
PA^(a) = 

By simple computations we have 


^'(A) = 


J2PA(&)9ia)e-^^^^ 
q£ -A 
(A) 




^"(A) = 

E PA(a)PA{b) ~ e-Mg(tt)+.(ii)} 


a.,bGA 




= - E pT^P-^pT^-^ 


a.bGA 


= - E^a 

aG^ 


2 

E^A^(^)5(a) 

a€A 


-\ 2 


We next prove the part c). Let A G [0,1]. By the Taylor 
expansion of (g|pxy) with respect to A around A = 0, 

we have 

= a^) = m+em+leio)irxf 


= E„ 


0;^“’,';^ {X,Y\U) 

Qx\\px,qxYiu^ ’ ' ' 




w 


(a.M) 

qx\\px,qxYiu 


{X,Y\U) 


for some r G [0,1]. Then by the dehnition of p, we have 

^A,PX^q\pxy) 


> AEq 


0;^“’,';^ {X,Y\U) 

qx\\px,qxY\u^ ’ ' \ 


- 2'’'“- 


The second inequality is obvious from the hrst inequality. 
We hnally prove the part d). By the hyperplane expression 
T^shiPxv) of TZ{pxy) stated Property |2] part b) we have that 
when (i?i + r, i ?2 + t) ^ TZ{pxy), we have 

/ 2 *(i?i+r)+p*(i ?2 + r) (61) 

for some p* G [0,1]. Let ao = ao(|T’|, |3^|) be the quantity 
dehned by (|7]i in Property |2] part b). Then from (IhTT i. we have 


/i*i?i+/r*i?2 + T<i?(^*)(pxy) 

(a) 1 
< 

a 


[a ( a\ I 

ClW-log C 2 - X - 
V a \ a/ T 


= Cl 


T2 / 1 _ 

■■ log I C 2 -— I —>■ 0 as T —>• 0. 


'Jl — 


r2+5 


Hence there exists a positive i/ = i^{S,\X\,\y\) with i/ < 
such that for r G (0, z/] C (0, ctg’*"’ ], 

fa ^ / a\ T 

Cl • W-log C 2 - < -. 

V Of \ a/ 2 


The above inequality together with 

(0,H, 


(63) 

yields that for r G 


fl*Ri+p*R2 + ir < (64) 

Then for each positive r with t G (0, i/j, we have the following 
chain of inequalities: 

F(i?i, i?2 |pxy) 

> supFl^^^''’'"*’^l(/2*i?i + p*i?2|pxy) 


A>0 


= sup 
A>0 
(a) 

> 


- Xt^+^{p*Ri + P*R2) 


(b) 

> 


1 + A(1 + /X*) 

-AT2+^(p*i?l +M*i?2)} 

1 / 1 \2 I 1 3+5 \ \ (c) P 2 f ' 

SSTT2a{- 2'’^ +2" T = 5n- 




2p 


Step (a) follows from Property|3]part b). Step (b) follows from 
(l64l i. Step (c) follows from an elementary computation. This 
completes the proof of Property [3 part d). ■ 

E. Proof of Lemma [7] 

To prove Lemma [T] we prepare a lemma. Set 

An = { (s,x",y") : -log ^'" ; >-p\- 


n qvr^{s,x^,y^) 


Furthermore, set 


^ A f n 1 1 PXr^ix'^) ^ ^ 

Bn= {x : - log - > -P } , 


n qx^{x'^) 

Bn=Bn^MiX y^,Bf = BfxMiX y^, 

Cn = {(s,a:") : s = p^f^\x^), 

Px^\six^\s) < Mie^^PxAxn}, 

Cn=CnXy^,Cf=Cfxy^, 


Vn = {(s,a:",y") : s = 


_^(a.+ ciW “ log (c 2 -) . (62) 

a \ a \ a/ 

Step (a) follows from the hrst inequality of (|6ll in Property |2] 

part b). Fix any (5 > 0. We choose a = . For this choice 

of a, we have (l62l i for any r G (0, ] and 


. o _ ,„(")/ 


Pf"|s(2/”|s) > (l/M2)e-"n, 


£n = {(s,a:",2/”) : s = p['"\x"), 

ij;(A(^ipA\x^),(p^f'\y^)) =2/”}. 

Then we have the following lemma. 

Lemma 12: 

PSX^Y^ [AD < e--\psx^Y^ {Bf) < e-"^ 
PSX^Y^ {CD < e-^fpsx^Yr^ (2?n nfn) < 
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Proof: We first prove the first inequality. 


Proof of Lemma Q} By definition we have 


Psx^yAAI) = Y. PvAs^x^y^) 

= e-”’'gy„ {Af) < e-"’'. 

Step (a) follows from the definition of An- On the second 
iniquality we have 

PSX^Y''{BD = PX”-{Bn) = E PXnix'^) 

< E 

= e-^^qx^ (bi) < 

Step (a) follows from the definition of Bn- We next prove the 
third inequality. 

PSX''Y’'{Cn) =PSX^{Cn) 

= E E PXAX^) 

Pxn (a:")<(l/Mi)e“'*'' 

xpx™|s(a;’*|s) 


< J-e-"’' V 

“Ml ^ 


E Px"|s(a;”|s) 

sGXii x^ :(pf'\x‘^)—s 

Pxn (x")<(l/Mi)e~”’> 


xpx-"|s(a:"|s) 


“ Ml ' ' 


psx^Y^ {An n n c„ n x>„) 

fl, <7sx^yn(^,X",F") ^ 

= PSX^Y^ \ - log- . . > -77, 

[n psx^y--{S,X^,Y^) 

> -77> 

gx"(X") 

0 > - log-T— - p, 

n px^{XA 

1 , ^ 1 , PxMX^\S) 

- log Ml > - log-——- 77 , 

TT 77, ) 

- logMa > - log-- 77 

Then for any {q}^i \ satisfying 

-log|ki"'^|| < RiA = 1 , 2 , 


we have 


psx^Y" {An n n c„ n T>„) 

fl, ^ 

< PSX^Y^ i - log- , ' > - 77 , 

177 

77X"(X") 

0 > - log-- - 77 , 


77 

77 

i ?2 > - log ■ 


77 , 

77 


77 pyn|s(y"|5) 

Hence, it suffices to show 

< psx-Y^ {An n n c„ n Vn) 


+4e 


— nr} 


Finally we prove the fourth inequality. We first observe that 

ps{s)= E Px"{x'^), Px^|s(a^"|s) = \ 

X {x )—s 

We have the following chain of inequalities: 


PSX^Y^ {Rn O £n) 

= E E Px"|s(a;"|s) 

s€A1i s 

X E pyn|xn(j/"|a;") 

y''-.A"'Hs,Af'\vA)=v" 

Py"|s(!/"|s)<(l/7W2)e“"'' 


= E E Fy"|s(7/"|s) 

s^Mi y"-.A"'Hs,AfHv’"))=y" 

Pv"|s(y"|s)<(l/7^2)e“"’' 


< E 


sGMi 


1 


^-np 


{y":7^(-)(s,^("^(27"))=y"}| 


< E "’7M2 = e- 

seATi ^ 


Step (a) follows from that the number of 7/" correctly decoded 
does not exceed M 2 - ■ 


to prove Lemma [T] By definition we have 

{£n)- 

Then we have the following. 

=PSX^Y^ {£n) 

~ PSX^Y^ {An n Bn H Cn H T^n H £n^ 

+P5X-F" {[An n n C„ n Vnf n £n) 

< psx’^Y”- {An n s„ n c„ n Vn) 

+PSX"F" {An) +PSX"Y" {Bn) +PSX"Y" {Cn) 
+PSX^Y^ {Rn £n) 

< psx^Y^ {An n n c„ n Vn) + 4e "'^- 

Step (a) follows from Lemma [121 ■ 

F. Proof of Lemma |5] 

In this appendix we prove Lemma |3 
Proof of Lemma |5} We first prove the following Markov 
chain ( [T3l l in Lemma [3 

SX^-^ ^ Xt^ Yt- 
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We have the following chain of inequalities: 


Steps (a) and (b) follow from ( l65l l. From (l66T l, we have 


IiYuSX*-^\Xt) = H{Yt\Xt) - H{Yt\SX*-^Xt) 

< H{Yt\Xt) - H{Yt\SX’^) H{Yt\Xt) - HiYt\X’^) 
H{Yt\Xt)-H{Yt\Xt)=0. 

Step (a) follows from that S = (p[^\x^) is a function 
of X". Step (b) follows from the memoryless property of 
the information source {(Xj, Yi)}^ 2 - Next we prove the 
following Markov chain (fT4l i in Lemma [2 

o s'x*-^ o {Xt,Yt). 

We have the following chain of inequalities: 

/(x*rt;y*-i|5x*-i) 

= iF(y‘"VX*-^) - H{Y*-^\SX*-^XtYt) 

< H{Y*-^\X*-^) - H{Y*-^\X’^SYt) 
iF(y‘-i|X*-^) - H(Y*-^\X^Yt) 
iF(y‘-i|X*-^) - H(Y*-^\X*-^Yt) = 0. 

Step (a) follows from that S = (f^^\x'^) is a function of 
X". Step (b) follows from the memoryless property of the 
information source {(Xi,■ 


G. Proof of Lemma |6] 

In this appendix we prove Lemma |6] 

Proof of Lemma^: By the definition of (s, ?/*), 

for t = 1, 2, • • •, n, we have 

(a,^,A)/ t t\ 

= Cf^psxtYtis,x\y*) 


n/, 




(65) 


Then we have the following chain of equalities: 

— Cf^psxtYt{s,x*,y*) 


n/< 

2=1 




= Ct \ y* 

2 = 1 

xPXtFtisx'-ir*-! {xt,yt\s, y‘“^) 
Cf^Ct-ip^sT^’%,-^ {s, y‘-i) 

>^PXtYt\sx*-^Yt-^{xt,yt\s,x* ^, 2 /‘ 

>^PXtYt\SX*-'^Yt-^{xt,yt\s,X* ^,J/‘ 


X/ 

i 


(a.p.A) 

gXt\\pXt,PXtYt\Ut 


{xt,yt\ut). 


- P^sxt-^Yt-^ a:‘"\ y‘"^) 

xpxtFt|sx*-iY*-i {xt,yt\s, x*~^, 

X y(a,p,A) (xt,yt\ut). 

qXtWPXt^PXtYtlUt^ ^ 




(67) 


( 68 ) 


Taking summations of (|67] | and dMt with respect to s,x*,y^, 
we obtain 


$ 


(a,A*,A) 


Pfx^’%^-^(s,x* \y* 1) 


s,x^ ,y^ 


XPXtYt\SX*-^Y<^-^ixt,yt\s,X* \y‘ 
X y(a,p,A) (xt,yt\ut)-, 

•'qXt\\PXt,PXtYt\Ut^ 


completing the proof. ■ 
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